<!DOCTYPE html>
<html lang="en">
<head>
	<meta charset="UTF-8">
	<meta http-equiv="X-UA-Compatible" content="IE=edge">
    <meta name="viewport" content="width=device-width, initial-scale=1">
	<title>Delimited payload token filter | ElasticSearch 7.7 权威指南中文版</title>
	<meta name="keywords" content="ElasticSearch 权威指南中文版, elasticsearch 7, es7, 实时数据分析，实时数据检索" />
    <meta name="description" content="ElasticSearch 权威指南中文版, elasticsearch 7, es7, 实时数据分析，实时数据检索" />
    <!-- Give IE8 a fighting chance -->
    <!--[if lt IE 9]>
    <script src="https://oss.maxcdn.com/html5shiv/3.7.2/html5shiv.min.js"></script>
    <script src="https://oss.maxcdn.com/respond/1.4.2/respond.min.js"></script>
    <![endif]-->
	<link rel="stylesheet" type="text/css" href="../static/styles.css" />
	<script>
	var _link = 'analysis-delimited-payload-tokenfilter.html';
    </script>
</head>
<body>
<div class="main-container">
    <section id="content">
        <div class="content-wrapper">
            <section id="guide" lang="zh_cn">
                <div class="container">
                    <div class="row">
                        <div class="col-xs-12 col-sm-8 col-md-8 guide-section">
                            <div style="color:gray; word-break: break-all; font-size:12px;">原英文版地址: <a href="https://www.elastic.co/guide/en/elasticsearch/reference/7.7/analysis-delimited-payload-tokenfilter.html" rel="nofollow" target="_blank">https://www.elastic.co/guide/en/elasticsearch/reference/7.7/analysis-delimited-payload-tokenfilter.html</a>, 原文档版权归 www.elastic.co 所有<br/>本地英文版地址: <a href="../en/analysis-delimited-payload-tokenfilter.html" rel="nofollow" target="_blank">../en/analysis-delimited-payload-tokenfilter.html</a></div>
                        <!-- start body -->
                  <div class="page_header">
<strong>重要</strong>: 此版本不会发布额外的bug修复或文档更新。最新信息请参考 <a href="https://www.elastic.co/guide/en/elasticsearch/reference/current/index.html" rel="nofollow">当前版本文档</a>。
</div>
<div id="content">
<div class="breadcrumbs">
<span class="breadcrumb-link"><a href="index.html">Elasticsearch Guide [7.7]</a></span>
»
<span class="breadcrumb-link"><a href="analysis.html">Text analysis</a></span>
»
<span class="breadcrumb-link"><a href="analysis-tokenfilters.html">Token filter reference</a></span>
»
<span class="breadcrumb-node">Delimited payload token filter</span>
</div>
<div class="navheader">
<span class="prev">
<a href="analysis-decimal-digit-tokenfilter.html">« Decimal digit token filter</a>
</span>
<span class="next">
<a href="analysis-dict-decomp-tokenfilter.html">Dictionary decompounder token filter »</a>
</span>
</div>
<div class="section">
<div class="titlepage"><div><div>
<h2 class="title">
<a id="analysis-delimited-payload-tokenfilter"></a>Delimited payload token filter<a class="edit_me edit_me_private" rel="nofollow" title="Editing on GitHub is available to Elastic" href="https://github.com/elastic/elasticsearch/edit/7.7/docs/reference/analysis/tokenfilters/delimited-payload-tokenfilter.asciidoc">edit</a>
</h2>
</div></div></div>

<div class="warning admon">
<div class="icon"></div>
<div class="admon_content">
<p>The older name <code class="literal">delimited_payload_filter</code> is deprecated and should not be used
with new indices. Use <code class="literal">delimited_payload</code> instead.</p>
</div>
</div>
<p>Separates a token stream into tokens and payloads based on a specified
delimiter.</p>
<p>For example, you can use the <code class="literal">delimited_payload</code> filter with a <code class="literal">|</code> delimiter to
split <code class="literal">the|1 quick|2 fox|3</code> into the tokens <code class="literal">the</code>, <code class="literal">quick</code>, and <code class="literal">fox</code>
with respective payloads of <code class="literal">1</code>, <code class="literal">2</code>, and <code class="literal">3</code>.</p>
<p>This filter uses Lucene’s
<a href="https://lucene.apache.org/core/8_5_1/analyzers-common/org/apache/lucene/analysis/payloads/DelimitedPayloadTokenFilter.html" class="ulink" target="_top">DelimitedPayloadTokenFilter</a>.</p>
<div class="note admon">
<div class="icon"></div>
<div class="admon_content">
<h3>Payloads</h3>
<p>A payload is user-defined binary data associated with a token position and
stored as base64-encoded bytes.</p>
<p>Elasticsearch does not store token payloads by default. To store payloads, you must:</p>
<div class="ulist itemizedlist">
<ul class="itemizedlist">
<li class="listitem">
Set the <a class="xref" href="term-vector.html" title="term_vector"><code class="literal">term_vector</code></a> mapping parameter to
<code class="literal">with_positions_payloads</code> or <code class="literal">with_positions_offsets_payloads</code> for any field
storing payloads.
</li>
<li class="listitem">
Use an index analyzer that includes the <code class="literal">delimited_payload</code> filter
</li>
</ul>
</div>
<p>You can view stored payloads using the <a class="xref" href="docs-termvectors.html" title="Term vectors API">term vectors API</a>.</p>
</div>
</div>
<div class="section">
<div class="titlepage"><div><div>
<h3 class="title">
<a id="analysis-delimited-payload-tokenfilter-analyze-ex"></a>Example<a class="edit_me edit_me_private" rel="nofollow" title="Editing on GitHub is available to Elastic" href="https://github.com/elastic/elasticsearch/edit/7.7/docs/reference/analysis/tokenfilters/delimited-payload-tokenfilter.asciidoc">edit</a>
</h3>
</div></div></div>
<p>The following <a class="xref" href="indices-analyze.html" title="Analyze API">analyze API</a> request uses the
<code class="literal">delimited_payload</code> filter with the default <code class="literal">|</code> delimiter to split
<code class="literal">the|0 brown|10 fox|5 is|0 quick|10</code> into tokens and payloads.</p>
<div class="pre_wrapper lang-console">
<pre class="programlisting prettyprint lang-console">GET _analyze
{
  "tokenizer": "whitespace",
  "filter": ["delimited_payload"],
  "text": "the|0 brown|10 fox|5 is|0 quick|10"
}</pre>
</div>
<div class="console_widget" data-snippet="snippets/911.console"></div>
<p>The filter produces the following tokens:</p>
<div class="pre_wrapper lang-text">
<pre class="programlisting prettyprint lang-text">[ the, brown, fox, is, quick ]</pre>
</div>
<p>Note that the analyze API does not return stored payloads. For an example that
includes returned payloads, see
<a class="xref" href="analysis-delimited-payload-tokenfilter.html#analysis-delimited-payload-tokenfilter-return-stored-payloads" title="Return stored payloads">Return stored payloads</a>.</p>
</div>

<div class="section">
<div class="titlepage"><div><div>
<h3 class="title">
<a id="analysis-delimited-payload-tokenfilter-analyzer-ex"></a>Add to an analyzer<a class="edit_me edit_me_private" rel="nofollow" title="Editing on GitHub is available to Elastic" href="https://github.com/elastic/elasticsearch/edit/7.7/docs/reference/analysis/tokenfilters/delimited-payload-tokenfilter.asciidoc">edit</a>
</h3>
</div></div></div>
<p>The following <a class="xref" href="indices-create-index.html" title="Create index API">create index API</a> request uses the
<code class="literal">delimited-payload</code> filter to configure a new <a class="xref" href="analysis-custom-analyzer.html" title="Create a custom analyzer">custom
analyzer</a>.</p>
<div class="pre_wrapper lang-console">
<pre class="programlisting prettyprint lang-console">PUT delimited_payload
{
  "settings": {
    "analysis": {
      "analyzer": {
        "whitespace_delimited_payload": {
          "tokenizer": "whitespace",
          "filter": [ "delimited_payload" ]
        }
      }
    }
  }
}</pre>
</div>
<div class="console_widget" data-snippet="snippets/912.console"></div>
</div>

<div class="section">
<div class="titlepage"><div><div>
<h3 class="title">
<a id="analysis-delimited-payload-tokenfilter-configure-parms"></a>Configurable parameters<a class="edit_me edit_me_private" rel="nofollow" title="Editing on GitHub is available to Elastic" href="https://github.com/elastic/elasticsearch/edit/7.7/docs/reference/analysis/tokenfilters/delimited-payload-tokenfilter.asciidoc">edit</a>
</h3>
</div></div></div>
<div class="variablelist">
<dl class="variablelist">
<dt>
<span class="term">
<code class="literal">delimiter</code>
</span>
</dt>
<dd>
(Optional, string)
Character used to separate tokens from payloads. Defaults to <code class="literal">|</code>.
</dd>
<dt>
<span class="term">
<code class="literal">encoding</code>
</span>
</dt>
<dd>
<p>(Optional, string)
Datatype for the stored payload. Valid values are:</p>
<div class="variablelist">
<dl class="variablelist">
<dt>
<span class="term">
<code class="literal">float</code>
</span>
</dt>
<dd>
(Default) Float
</dd>
<dt>
<span class="term">
<code class="literal">identity</code>
</span>
</dt>
<dd>
Characters
</dd>
<dt>
<span class="term">
<code class="literal">int</code>
</span>
</dt>
<dd>
Integer
</dd>
</dl>
</div>
</dd>
</dl>
</div>
</div>

<div class="section">
<div class="titlepage"><div><div>
<h3 class="title">
<a id="analysis-delimited-payload-tokenfilter-customize"></a>Customize and add to an analyzer<a class="edit_me edit_me_private" rel="nofollow" title="Editing on GitHub is available to Elastic" href="https://github.com/elastic/elasticsearch/edit/7.7/docs/reference/analysis/tokenfilters/delimited-payload-tokenfilter.asciidoc">edit</a>
</h3>
</div></div></div>
<p>To customize the <code class="literal">delimited_payload</code> filter, duplicate it to create the basis
for a new custom token filter. You can modify the filter using its configurable
parameters.</p>
<p>For example, the following <a class="xref" href="indices-create-index.html" title="Create index API">create index API</a> request
uses a custom <code class="literal">delimited_payload</code> filter to configure a new
<a class="xref" href="analysis-custom-analyzer.html" title="Create a custom analyzer">custom analyzer</a>. The custom <code class="literal">delimited_payload</code>
filter uses the <code class="literal">+</code> delimiter to separate tokens from payloads. Payloads are
encoded as integers.</p>
<div class="pre_wrapper lang-console">
<pre class="programlisting prettyprint lang-console">PUT delimited_payload_example
{
  "settings": {
    "analysis": {
      "analyzer": {
        "whitespace_plus_delimited": {
          "tokenizer": "whitespace",
          "filter": [ "plus_delimited" ]
        }
      },
      "filter": {
        "plus_delimited": {
          "type": "delimited_payload",
          "delimiter": "+",
          "encoding": "int"
        }
      }
    }
  }
}</pre>
</div>
<div class="console_widget" data-snippet="snippets/913.console"></div>
</div>

<div class="section">
<div class="titlepage"><div><div>
<h3 class="title">
<a id="analysis-delimited-payload-tokenfilter-return-stored-payloads"></a>Return stored payloads<a class="edit_me edit_me_private" rel="nofollow" title="Editing on GitHub is available to Elastic" href="https://github.com/elastic/elasticsearch/edit/7.7/docs/reference/analysis/tokenfilters/delimited-payload-tokenfilter.asciidoc">edit</a>
</h3>
</div></div></div>
<p>Use the <a class="xref" href="indices-create-index.html" title="Create index API">create index API</a> to create an index that:</p>
<div class="ulist itemizedlist">
<ul class="itemizedlist">
<li class="listitem">
Includes a field that stores term vectors with payloads.
</li>
<li class="listitem">
Uses a <a class="xref" href="analysis-custom-analyzer.html" title="Create a custom analyzer">custom index analyzer</a> with the
<code class="literal">delimited_payload</code> filter.
</li>
</ul>
</div>
<div class="pre_wrapper lang-console">
<pre class="programlisting prettyprint lang-console">PUT text_payloads
{
  "mappings": {
    "properties": {
      "text": {
        "type": "text",
        "term_vector": "with_positions_payloads",
        "analyzer": "payload_delimiter"
      }
    }
  },
  "settings": {
    "analysis": {
      "analyzer": {
        "payload_delimiter": {
          "tokenizer": "whitespace",
          "filter": [ "delimited_payload" ]
        }
      }
    }
  }
}</pre>
</div>
<div class="console_widget" data-snippet="snippets/914.console"></div>
<p>Add a document containing payloads to the index.</p>
<div class="pre_wrapper lang-console">
<pre class="programlisting prettyprint lang-console">POST text_payloads/_doc/1
{
  "text": "the|0 brown|3 fox|4 is|0 quick|10"
}</pre>
</div>
<div class="console_widget" data-snippet="snippets/915.console"></div>
<p>Use the <a class="xref" href="docs-termvectors.html" title="Term vectors API">term vectors API</a> to return the document’s tokens
and base64-encoded payloads.</p>
<div class="pre_wrapper lang-console">
<pre class="programlisting prettyprint lang-console">GET text_payloads/_termvectors/1
{
  "fields": [ "text" ],
  "payloads": true
}</pre>
</div>
<div class="console_widget" data-snippet="snippets/916.console"></div>
<p>The API returns the following response:</p>
<div class="pre_wrapper lang-console-result">
<pre class="programlisting prettyprint lang-console-result">{
  "_index": "text_payloads",
  "_type": "_doc",
  "_id": "1",
  "_version": 1,
  "found": true,
  "took": 8,
  "term_vectors": {
    "text": {
      "field_statistics": {
        "sum_doc_freq": 5,
        "doc_count": 1,
        "sum_ttf": 5
      },
      "terms": {
        "brown": {
          "term_freq": 1,
          "tokens": [
            {
              "position": 1,
              "payload": "QEAAAA=="
            }
          ]
        },
        "fox": {
          "term_freq": 1,
          "tokens": [
            {
              "position": 2,
              "payload": "QIAAAA=="
            }
          ]
        },
        "is": {
          "term_freq": 1,
          "tokens": [
            {
              "position": 3,
              "payload": "AAAAAA=="
            }
          ]
        },
        "quick": {
          "term_freq": 1,
          "tokens": [
            {
              "position": 4,
              "payload": "QSAAAA=="
            }
          ]
        },
        "the": {
          "term_freq": 1,
          "tokens": [
            {
              "position": 0,
              "payload": "AAAAAA=="
            }
          ]
        }
      }
    }
  }
}</pre>
</div>
</div>

</div>
<div class="navfooter">
<span class="prev">
<a href="analysis-decimal-digit-tokenfilter.html">« Decimal digit token filter</a>
</span>
<span class="next">
<a href="analysis-dict-decomp-tokenfilter.html">Dictionary decompounder token filter »</a>
</span>
</div>
</div>

                  <!-- end body -->
                        </div>
                        <div class="col-xs-12 col-sm-4 col-md-4" id="right_col">
                        
                        </div>
                    </div>
                </div>
            </section>
        </div>
    </section>
</div>
<script src="../static/cn.js"></script>
</body>
</html>